Intelligent Testing With Torrance

Applying the idea of “intelligent testing” (Kaufman, 1979, 1994) to creativity assessment could broaden our understanding of creativity assessment and would be in accord with the vision of E. Paul Torrance. Such intelligent creativity testing would not seek a single creativity score. Testers would consider patterns of scores in different domains and interpret them for comparative strengths and weaknesses in various areas of creative activity. Specific suggestions on how this might be done are offered. From 1974 to 1979, two leaders in the field of psychological assessment were both at the University of Georgia: Dr. E. Paul Torrance and Dr. Alan S. Kaufman. Despite their proximity (and personal friendship), they never collaborated on a project together. The goal of this article is toapplyKaufman’s ideasabout intelligent testing to the Torrance Tests and Torrance’s work and to suggest how the integration of Torrance and Kaufman might representastepforwardfor thefieldofcreativity. For many years, the point of being assessed with an IQ test was to get one, two, or three scores—Verbal, Performance, and Full-Scale IQs—that were often treated as magic numbers that could open or shut doors of opportunity. If one or more IQs were above a certain point, better school or work opportunities might await. If IQs were below a certain point, someone might be eligible for federal support or assistance. Creativity tests have no such power. They are sometimes used with gifted programs, but the impact of scoring well on a measure of creativity is quite less than that of scoring well on an IQ test. Yet creativity assessments, like IQ tests, are geared toward global numbers or scores. A popular philosophy of IQ testing, however, that disdains global scores and has had a tremendous influence on the field is that of “intelligent testing” (Kaufman, 1979, 1994). Using this system, the tester is elevated above the test. The global scores mean little by themselves.Thekey is interpreting thescores incontext. The persons administering the test are expected to use their qualifications and training and to bring their own experience to the testing session. In this manner, the tester can help the child or adult being tested by understanding and interpreting a wide range of behaviors, making inferences about any observed problem-solving strategies, and directly applying the latest theories and research results directly to the person’s specific set of scores. Every aspect of psychology is brought into play to interpret a profile of scores in the context of accumulated research, theory, and clinical practice. This profile is used to help solve problems and create solutions for the person tested—that is, providing answers to the referral questions—not merely as a label or classification system (Kaufman, 1979, 1994). We believe this approach can be applied to creativity research. A qualified tester would be well versed in the fields of social, cognitive, and educational psychology (among others). The pattern of scores in the different domains could be interpreted for its comparative strengths and weaknesses. Rather than merely producing a single number that is of little use to a student, this new domain-specific creativity could help students disCreativity Research Journal 2006, Vol. 18, No. 1, 99–102 Copyright © 2006 by Lawrence Erlbaum Associates, Inc. Creativity Research Journal 99 The authors would like to thank Alan S. Kaufman and Scott Barry Kaufman for helpful comments and suggestions on an earlier draft. Correspondence and requests for reprints should be sent to James C. Kaufman, Learning Research Institute, California State University at San Bernardino, Department of Psychology, 5500 University Parkway, San Bernardino, CA 92407. E-mail: [email protected] cover and validate areas of creative talent in themselves. In addition, an administrator using the intelligent testing approach could look for signs of insufficient motivation, a thinking style that might conflict with the task, or other areas that could be improved for enhanced creative potential. This concept, we believe, would be in line with Torrance’s original aims in the development of his divergent thinking tests (the Torrance Tests of Creative Thinking or TTCT; Torrance, 1966, 1974). Torrance did not design his tests for the use to which they are most commonly put these days—identification of students for gifted/talented programs (Kim, 2006). His primary goals in developing these tests were to help us better understand the human mind and its functioning; to find ways to better individualize instruction, including remedial and psychotherapeutic interventions; to evaluate the effectiveness of educational programs; and to become more sensitive to latent potential in people. With these goals in mind, his first tests provided subscores in the areas of fluency, flexibility, originality, and elaboration (Torrance, 1966, 1974) based on the earlier work on divergent production of Guilford (1956, 1967; Guilford & Hoepfner, 1971). This stimulated a great deal of research, some of which did not support the independence of these four subtests. Hocevar’s analyses (1979a, 1979b), for example, suggested that fluency was the only variable of the four actually being measured; similarly, Dixon (1979) showed that originality scores were so dependent on fluency scores as to make them redundant. Factor analytic studies (e.g., Clapham, 1998; Heausler & Thompson, 1988) have also sometimes suggested that the TTCT measures a single factor, and Treffinger (1986) argued that the available evidence did not warrant the use of these four subscale scores as independent measures. Regardless of any structural or conceptual criticisms of the TTCT, there are many studies that lend support for its validity. Torrance (1972a, 1972b, 1990) reported more than a dozen studies by several researchers showing shortand long-term predictive validity of the tests. These studies included subjects ranging in age from kindergarten through adults. Torrance and Safter (1989), for example, used one of the subtests (Just Suppose) to conduct a long-range study looking at predictive validity and found a solid relationship after 20 years to creative achievement. Plucker (1999) reanalyzed data from a different Torrance longitudinal study and found that divergent thinking accounted for three times more variance in creative achievement than traditional IQ tests. The validity of these confirmatory studies has, in turn, been challenged (Baer, 1993; Crockenberg, 1972), just as studies showing no correlation between the tests and later creative accomplishment have been found wanting (Plucker, 1999; Torrance, 1972a). It appears fair to conclude that even if perfect consensus has been elusive regarding the validity of the TTCT, there is no question that they are the most widely used tests of creativity (Baer, 1993; Kim, 2006; Torrance & Presbury, 1984). In more recent editions of the TTCT, Torrance changed the scoring procedures and introduced 15 new measures: 2 norm-referenced and 13 criterion-referenced (Torrance, 1990, 1998). For example, in addition to the original norm-referenced subtests of fluency, originality, and elaboration (flexibility has been dropped), measures of abstractness of titles and resistance to premature closure have been added to the scoring of the figural test. These new measures open the possibility of more intelligent creativity testing if the various subtests prove to be valid and reliable measures. One component of intelligent testing is analyzing subtests to look for strengths and weakness and patterns of abilities. The newer TTCT also may offer this promise. We find it interesting—perhaps even prophetic—that the TTCT has for many decades come in two forms, verbal and figural, which suggests that Torrance was long aware of a need to look at how creativity may be different in different domains. The issue of domain specificity has in the past decade become an increasingly important one in creativity theory and research (Baer, 1993, 1998; Kaufman & Baer, 2004, 2005a; Plucker, 1998, 2005; Plucker & Beghetto, 2004). An effort to include both domain-general and domain-specific aspects of creativity is a primary motivation for the theory of creativity that we have been developing, the APT model (Baer & Kaufman, 2005a; Kaufman & Baer, 2005a, 2005b, in press). It is a model that would allow testing of creativity-relevant skills and interests both in different domains and at different levels of domain specificity or generality. This kind of theory-based testing could provide a rich individual profile of a test taker’s creative abilities and interests; in doing so, it would move forward Paul Torrance’s (1966, 1974) original goals of helping us 100 Creativity Research Journal J. C. Kaufman and J. Baer